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1.  INTRODUCTION 


Contemporary  mathematical  programming  models  are  often  so  large 
that  direct  solution  of  the  associated  linear  programming  (LP)  problems 
with  the  classical  simplex  method  is  prohibitively  expensive,  if  not 
impossible  in  a  practical  sense.  It  has  been  found  that  most  of  these 
problems  are  sparse,  with  relatively  few  non-zero  coefficients,  and 
usually  possess  very  systematic  structure .  These  problems  exhibit  inherent 
structural  characteristics  that  can  be  exploited  by  specializations  of 
the  simplex  procedure.  Various  types  of  regularity  are  often  described 
as,  for  instance,  block  angular,  staircase,  and  so  forth;  terms  all 
chosen  to  describe  the  visual  appearance  of  the  non- zero  ceofficients 
when  the  rows  and  colunns  of  the  problem  are  suitably  ordered.  There 
are  profound  economic,  managerial  and  mathematical  motives  for  this 
special  structure,  which  are  not  examined  here. 

Methods  to  exploit  special  model  structure  can  be  categorized 
generally  as  ■LndUiZ.dt  (e.g.,  decomposition),  where  a  solution  to  the 
original  problem  is  achieved  by  dealing  with  related  models  which  are 
Individually  easier  to  solve,  or  as  dLOtzet,  when  the  original  problem  is 
solved  by  a  modified  simplex  algorithm. 

Among  the  direct  methods,  the  most  frequently  used  technique  is 
called  ba&'CA  factorization  [6],  where  the  reflection  of  special  problem 
structure  appears  and  is  used  to  good  benefit  in  the  intermediate  LF 
bases.  Basis  factorization  can  be  dynamic,  where  the  algorithm  deals 
with  each  basis  sequentially  and/or  independently  in  an  attempt  to  extract 
as  much  specialized  basis  structure  as  possible,  or  static,  where  the 
algorithm  depends  upon  certain  types  of  special  structure  being 
present  in  all  bases. 
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Static  basis  factorizations  include  *impZz  uppzA  bound*, 
gZMJuxLizzd  upper.  bound*  (GUB) ,  and  embedded  network  AXM* ,  among  many 
others.  Simple  upper  bounds  are  a  set  of  rows  for  which  each  row  has 
only  one  non-zero  coefficient.  Generalized  upper  bounds  are  a  set  of 
rows  for  which  each  column  (restricted  to  those  rows)  has  at  most  one 
non-zero  coefficient.  Network  rows  are  a  set  of  rows  for  which  each 
column  (restricted  to  those  rows)  has  at  most  two  non-zero  coefficients 
of  opposite  sign. 

Each  of  these  factorizations  permits  the  simplex  algorithm  to 
deal  with  the  static  subsets  of  the  rows  (and  columns)  of  all  bases 
encountered  with  prior  knowledge  that  they  will  satisfy  very  restricted 
rules.  Most  of  these  methods  work  best  when  Zogic  can  be  substituted 
for  arithmetic  (as  is  the  case  with  the  coefficients  +  1) .  For  this 
reason,  static  factorizations  often  restrict  the  special  structure  to 
possess  only  +  1,  or  to  be  4C atid  so  as  to  produce  an  equivalent  result. 

The  concept  of  generalized  upper  bounds  was  introduced  in  1964, 
the  result  of  work  by  Dantzig  and  Van  Slyke  [4].  The  name  is  derived 
from  analogy  to  the  simple  upper  bound  structure.  Graves  and  McBride  [6] 
refer  to  Static.  Signtd  Identity  Factorization  as  a  term  more  suggestive 
of  the  implied  basis  structure.  Since  their  introduction,  some  form  of 
GtJB  has  been  Implemented  in  many  commercial  LP  systems.  There  is  often 
confusion  between  the  mathematical  characterization  of  GUB  and  these 
various,  widely  used  implementations  of  GUB,  in  that  the  latter  often 
restrict  the  GUB  set  membership  rules  to  permit  uncomplicated  simplex 
logic.  All  of  the  methods  reported  here  address  the  full  generality  of 
GUB  sets  but  can  be  modified  to  produce  restricted  GUB  sets  as  necessary. 
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The  details  of  how  GUB  can  be  exploited  to  reduce  the  confutations 
of  the  simplex  algorithm  are  not  discussed  here.  See  [1,4,6,10,12].  The 
underlying  concept  is  that  the  GUB  structure  enables  the  simplex  algorithm 
to  manipulate  the  GUB  rows  implicitly,  with  logic  rather  than  floating 
point  arithmetic,  thus  reducing  the  effective  size  and  solution  time 
for  the  problem.  The  more  rows  one  is  able  to  GUB,  the  fewer  rows  one 
has  to  explicitly  carry  through  the  simplex  operations.  If  the  original 
problem  has  m  constraints  (of  which  p  are  GUB  rows)  and  n  variables, 
then  at  most  only  an  (m-p  x  m-p)  submatrix  of  the  basis  is  needed  for 
the  explicit  simplex  operations.  This  contracted  explicit  basis  means 
that  many  of  the  calculations  are  replaced  with  logical  operations, 
yielding  faster  results  and  less  numerical  rounding  error. 

Many  problem  types  have  natural  GUB  structures  embedded  in  them. 

a)  "Transportation  problems  (pure,  bounded  and  capacitated  networks) 

b)  Multi-product  blending 

c)  Raw  material  and/or  production  resources  allocation  (forest  management 
machine  loading,  plant  scheduling) 

d)  Operations  planning  (combined  production/inventory/distribution 
planning) 

e)  Resource  assignment  (i.e.,  freight  cars,  personnel)"  [14]. 

To  illustrate.  Figure  1  presents  a  transportation  type  problem. 
Note  that  a  GUB  set  has  been  marked.  For  problems  similar  to  this,  a 
large  GUB  set  can  be  found  quickly  by  visual  inspection.  Likewise,  in 
a  particular  &£o44  of  models,  knowledge  of  the  model  structure  can  lead 
to  problem-independent  specification  of  GUB  sets  (e.g.,  one  can  always 


GUB  all  the  sinks  of  a  pure  network) .  But  for  many  general  models  this 
technique  cannot  be  used,  and  due  to  the  large  size  of  the  problems, 
visual  Inspection  may  be  limited  to  adjacent  rows  or  patterns.  This 
Is  very  dependent  on  how  the  problem  Is  written.  Most  contemporary 
problems  for  which  GUB  factorization  may  be  crucial  are  so  large  that 
an  explicit  "picture"  of  the  coefficient  matrix  Is  impractical.  (One 
notorious  example  has  been  encountered  with  a  "picture"  measuring  10 
by  300  meters!)  It  is  therefore  highly  desirable  to  have  an  automatic 
procedure  whereby  a  GUB  set  can  be  identified  efficiently. 
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Detached  coefficient  array  for  a 
transportation  problem 

Figure  1 


In  large  problems  there  exists  a  huge  number  of  subsets  of  rows 
that  satisfy  the  GUB  criteria.  It  Is  generally  regarded  that  those  sub¬ 
sets  with  more  rows  are  "better"  GUB  sets  since  they  imply  a  more  con¬ 
tracted  explicit  basis.  The  Implied  problem,  then,  is  to  find  the  maximum 


GUB  set. 
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Integer  algorithms  to  find  a  maximum  GUB  set  do  exist. 

These  usually  entail  enumeration  schemes  and  cannot  be  guaranteed  to  be 
efficient  in  a  practical  sense.  Conceivably,  2m-m  sets  of  rows  might 
have  to  be  searched  before  a  maximum  GUB  structure  is  found.  As  the 
problem  size  grows,  the  number  of  possible  sets  that  need  to  be  checked 
increases  txponzntiaJity .  As  will  be  shown  later,  the  hope  of  finding 
an  efficient  algorithm  to  find  the  nuvUmum  GUB  set  for  any  general 
problem  is  dim. 

Therefore,  researchers  and  practitioners  have  concentrated  on 
constructing  efficient  he-UA^ii-tcC.  algorithms  that  attempt  to  identify, 
but  do  not  guarantee,  a  maximum  GUB  set.  A  few  of  these  methods  show¬ 
ing  great  promise  have  been  reported,  but  they  have  not  been  tested  with 
large  scale  problems. 

This  report  outlines  several  automatic  heuristic  GUB  finding  pro¬ 
cedures  that  have  been  developed  and  published  in  the  recent  literature . 
These  procedures  are  tested  on  a  suite  of  large  scale,  real  life  optimi¬ 
zation  problems,  and  are  modified  to  improve  their  behavior.  Comparative 
performance  of  the  methods  is  given  both  in  terms  of  the  computational 
effort  to  identify  a  GUB  set,  as  well  as  the  size  of  the  GUB  set 
achieved. 

Identification  of  GUB  sets  of  maximum  row  dimension  is  shown  in 
Section  7  to  be  among  the  class  of  NP-complete  problems.  However,  an 
easily  computed  uppZA.  bound,  on  the  size  of  the  maximum  GUB  set  is  developed 
and  used  to  evaluate  objectively  the  quality  of  heuristic  GUB  algorithms, 
showing  that  very  nearly  maximum  GUB  sets  are  routinely  achieved. 


2.  PROBLEM  DEFINITION  AND  REPRESENTATIONS 


The  Linear  Programming  problem  is  defined  as 


(L)  Min  cCx 

s .  t .  _<  Ax  £  r 

b  <  x  <  b 


(ranged  constraints) 
(simple  bounds) 


where  _r  and  r  are  m-vectors,  x,  c,  b^  and  b  are  n-vectors  and  A 
is  an  m  *  n  matrix.  The  constraints  are  sometimes  defined  as  equations, 
but  for  the  general  case  of  GUB  treated  here  constraints  can  be  equations, 
inequalities  or  a  mixture.  The  immediate  discussion  will  be  directed 
at  (L) ;  integer  and  mixed  integer  problems  are  treated  later. 

For  identification  of  a  GUB  set  of  rows,  only  the  matrix  A  is 
used  and  since  the  actual  values  of  the  non-zero  elements  of  the  matrix  A 
are  not  required  the  associated  matrix  K  is  defined. 


K  =  (k±j)  :  k 


0  if  a±j  -  0 


1  otherwise 


An  an  example,  consider  the  following  linear  programming  problem. 


Min  x^  + 

3*2 

-  *3  +  3x4  +  *5 

-2*g 

S.t.  *i  + 

2*2 

+  x3 
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4 

6*i 

+  2*5 

+  5*0 

- 

15 

“3*2 

+  2.7*3 

- 

7 

6*4 

+  4*0 

m 

8 

2*1 

+ 

H 

cn 

m 

1 

The  corresponding  matrices  A  and  K  are : 


A 


1 

6 

0 

0 

2 


2  10 

0  0  0 

-3  2.7  0 

0  0  6 

0  0  0 


0 

2 

0 

0 

1 


0 

5 

0 

4 
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There  are  several  ways  one  can  model  the  maximum  GUB  problem. 
Three  approaches  are  presented  to  aid  in  the  understanding  of  the 
theoretical  context  of  the  heuristic  methods  examined  and  to  highlight 
the  formal  complexity  of  the  original  problem. 

Two  rows  of  K  (or  A)  are  said  to  con^LLct  if  there  exists 
at  least  one  column  with  non-zero  coefficients  in  both  rows.  The  GUB 
problem  can  be  restated  as  that  of  finding  a  subset  of  the  rows  that 
do  not  conflict. 


Graph  Theory  Representation 

Consider  the  matrix  K  of  the  linear  programming  problem  (L) . 
A  graphical  representation  of  this  matrix  can  be  constructed  through 
the  following  mapping  rule,  f.  Let  each  row  of  K  be  a  vertex  of  the 
graph.  Should  two  rows  of  K  conflict  then  the  two  vertices  of  the 
graph  are  joined  by  an  edge.  This  mapping  retains  all  the  necessary 
conflict  information. 


The  graph  associated  with  the  example  is  presented  in  Figure  2. 
Note  that  it  has  five  vertices,  one  for  each  row.  Since  row  1  conflicts 
with  rows,  2,  3,  and  5,  edges  connect  vertex  1  to  those  vertices.  The 
other  edges  represent  the  remaining  conflicts.  If  two  vertices,  a  and  b 
are  joined  by  an  edge,  e,  then  a  and  b  are  adjacent,  and  a  (or  b) 
is  tnctdent  Moth  e.  Since  vertices  2  and  3  are  not  adjacent,  this 
indicates  that  the  corresponding  two  rows  in  K  do  not  conflict. 


Figure  2 


This  introduces  the  notion  of  lnde.pe.nde.nce.  Given  a  graph 
G  ■  (V,E),  a  subset  V'  €  V  is  said  to  be  an  independent  &et  if  no 
two  of  its  elements  are  adjacent.  It  follows  that  if  an  independent 
set  of  vertices  can  be  found  in  G  then  the  corresponding  rows  of  the 
matrix  K  do  not  conflict  and  thus  define  a  GUB  set.  Conversely,  a 
GOB  set  for  K  defines  an  independent  set  for  the  graph  G.  It  is  also 
clear  that  an  Independent  set  for  G  is  maximum  if  and  only  if  the 
corresponding  GUB  set  for  K  is  maximum. 
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In  the  example  problem,  the  maximum  independent  set  Is  3,  4,  5. 
These  are  also  the  rows  of  the  maximum  GUB  structure. 

Consider  the  set  K^,  the  set  of  all  K-type  matrices  having  m 
rows.  The  above  mapping  factors  this  set  into  a  finite  nusfcer  of  cZa66t&. 
Two  matrices,  and  are  said  to  belong  to  the  same  class,  C,  if 

and  only  if  each  is  mapped  into  the  same  graph,  G^. 


Thus,  an  independent  set  of  vertices  of  G ^  correspond  to  a  GUB  row  set 
for  every  matrix  in  the  class  C. 

The  incidence  matrix  N  is  defined  as  follows 


For  the  exas?le  problem  N  would  be 


if  vertex  i  is  incident  with  edge  j 
otherwise 


10 


N- 


V1 

v2 

V3 

v4 

*5 


•l 

1 

0 

1 

0 

0 


e2 

1 

0 

0 

0 

1 


e3 

1 

1 

0 

0 

0 


e4 

0 

1 

0 

0 

1 


There  exists  one,  and  only  one  Incidence  matrix  for  each  graph  of  G, 

where  G  Is  the  set  of  all  graphs  having  m  vertices. 

Since  the  set  of  all  N-type  matrices  with  m  rows  is  a  subset 

of  K  ,  every  class  of  K  contains  one  and  only  one  Incidence  matrix, 
m  m 

In  general,  for  the  GUB  problem,  every  m  row  matrix  is  equivalent 
to  one  of  a  finite  nus&er  of  incidence  matrices.  Superficially  this 
may  seem  to  be  a  simplfication.  But  as  shown  in  Section  7  the  GUB 
problem  on  N  is  as  difficult  as  the  independent  sec  problem  on  G. 

The  equivalent  statements  of  the  GUB  problem  do,  however,  offer 
different  views  of  the  problem  which  are  helpful  in  considering  algorithms 
for  and  analysis  of  the  problem.  [Note:  In  Garey  and  Johnson  [5]  it 
is  shown  that  two  other  graph  problems,  the  "vertex  cover"  and  the  "clique" 
problem,  are  equivalent  to  the  independence  problem,  and  hence  the  GUB 
problem.  These  problems  do  not  seem  to  offer  any  additional  insight 
for  the  GUB  problem.  ] 


Conflict  Matrix  Representation 

The  first  heuristic  approach  is  developed  around  a  COn^-LLct  ma&iLx. 
This  is  a  square  matrix  of  dimension  m,  defined  by: 
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1  if  row  i  conflicts  with  row  j  in  (L) 

0  otherwise  . 

For  the  example  problem: 

‘1110  1 
110  11 

M-  1  0  1  0  0 

0  10  1  0 
.110  0  1 

Note  that  this  matrix  is  symmetric.  The  sum  for  any  row  (or  column) 
indicates  the  number  of  other  rows  it  is  in  conflict  with,  plus  one. 

This  sum  is  important  in  that  it  indicates  for  any  particular 
row  how  many  other  rows  would  be  subsequently  excluded  from  a  GUB  set 
by  its  addition. 

The  rows  of  a  GOB  structure  can  be  rearranged  to  form  an 
embedded  identity  matrix  in  M.  Note  that  this  is  the  case  for  rows 
3,  4,  and  5  in  the  example  problem. 

Vector  Space  Representation 

The  second  heuristic  approach  can  be  modeled  using  vectors  in 
an  n-dimensional  vector  space,  where  n  is  the  number  of  variables  in 
the  problem  (L).  Consider  each  row  of  K  as  a  vector  in  this  space, 
having  unit  length  in  those  "dimensions"  corresponding  with  its  non-zero 
coefficients.  In  the  example  problem,  row  1  is  represented  by  the 
following  vector: 


t 

1 


(1,1,1, 0,0,0)  . 

R,  che  resultant  vector  from  the  sum  of  all  vectors  of  the  rows 
of  K,  indicates  the  number  of  conflicts,  plus  one,  associated  with  each 
variable  of  (L) .  A  hypercube  in  n-space  situated  in  the  first  orthant 
at  the  origin  with  length  1  in  all  positive  directions  denotes  the 
feasible  GUB  region.  Should  R  extend  beyond  this  are?,  then  the  set 
of  rows  corresponding  to  the  vectors  determining  R  does  not  constitute 
a  GOB  structure. 

A  gradient  vector  can  be  calculated  indicating  the  direction  of 
the  shortest  distance  to  the  fi&gion.  It  can  be  used  to  determine 

which  row  to  remove  from  the  set  to  obtain  the  largest  movement  on  the 
desired  direction.  When  R  falls  with in  the  feasible  region,  the  set 
of  rows  determining  R  constitutes  a  GUB  set. 
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3 .  EARLIER  LITERATURE 


Two  papers  dealing  with  efficient  GUB  finding  methods  are  worthy 
of  special  note. 

Brearley,  Mitra  and  Williams  [2]  establish  a  very  useful  framework 
for  study  of  methods  for  finding  GUB  structure,  as  well  as  an  insightful 
discussion  of  these  methods  and  a  taxonomy  for  their  classification. 

They  define  three  sets  consisting  of  the  rows  of  the  technological 
matrix  A.  The  first  set,  the  eJHgthte  6Zt,  is  made  up  of  every  row  of 
A  that  is  individually  eligible  to  belong  in  the  GUB  set.  The  itAuctuAZ 
6&t  is  a  subset  of  the  eligible  set  and  includes  all  those  rows  currently 
considered  as  members  of  the  GUB  set.  The  candidate.  4 It  consists  of 
those  rows  of  the  eligible  set  that  are  candidates  for  inclusion  (or  re- 
inclusion)  in  the  GUB  set.  Every  one  of  the  methods  examined  in  [2] 
is  described  in  terms  of  manipulation  of  these  sets. 

Each  method  of  building  a  GUB  set  employs  one  of  two  basic  strategies 
The  type.  I  (row  addition)  strategy  begins  with  an  empty  structure  set. 

Then,  based  on  a  particular  type  I  criteria  for  inclusion,  rows  are  removed 
from  the  candidate  set  and  either  added  to  the  structure  set  or  dropped 
from  further  consideration.  This  procedure  continues  until  the  candidate 
set  is  empty.  The  rows  in  the  structure  set  form  an  admissible  GUB 
structure. 

The  type.  77  (row  deletion)  strategy  takes  the  opposite  approach  and 
is  divided  into  two  phases.  Methods  of  this  type  initially  place  all 
eligible  rows  in  the  structure  set.  This  normally  leads  to  an  infeasible 
GUB  set  with  many  conflicting  rows.  Based  upon  the  particular  type  II 


decision  rules,  rows  are  removed  from  the  structure  set  and  placed  in 
the  candidate  set.  The  first  phase  of  this  strategy  ends  when  a  feasible 
structure  is  obtained. 

The  second  phase  involves  examining  the  removed  rows  in  the 
candidate  set.  Those  that  do  not  conflict  with  any  of  the  members  of 
the  current  structure  set  are  taken  from  the  candidate  set  and  reincluded 
in  the  structure  set.  Those  that  do  conflict  are  deleted  from  the  can¬ 
didate  set  and  dropped  from  further  consideration.  The  second  phase  ends 
when  the  candidate  set  is  empty.  At  this  point  the  rows  of  the  structure 
set  constitute  an  admissible  GUB  set. 

Brearley,  Mitra,  and  Williams  examine  over  18  different  methods. 

These  approaches  differ  in  the  primary  and  secondary  decision  criteria 
for  including  (or  removing)  a  row  in  the  GUB  structure  set.  The  heuristic 
decision  rules  examined  are  based  on  the  following  model  entities  and 
combinations  thereof: 

Include  or  remove  a  row  based  upon: 

a)  the  number  of  non-zero  elements  in  the  given  row, 

b)  the  number  cf  rows  in  conflict  with  the  given  row, 

c)  the  number  of  non-zero  elements  in  rows  that  conflict  with  the  given  row, 

d)  the  row's  relative  weight  obtained  by  the  inner  product  of  a  vector 
representation  of  the  row  and  a  directional  gradient. 

Type  I  methods,  using  all  these  rules  were  tested,  as  were  Type  II 
methods  using  rules  a,  b  and  c.  As  an  exao?le,  1.1  (type  I  strategy, 
method  number  1)  has  as  its  primary  decision  rule  for  adding  rows  to  the 
structure  set:  choose  a  row  from  the  candidate  set  with  the  minimum 
number  of  non-zero  elements.  Method  II. 1  removes  rows  from  the  structure 
set  based  upon  the  maximum  non- zero  element  count  for  each  row. 


I 
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These  methods  were  implemented  with  an  ALGOL  program  run  on 
an  ICL  4130  computer.  Twelve  linear  programming  problems  ranging  in 
size  from  12  rows  up  to  166  rows  were  used  for  computational  tests. 

The  results  show  that  those  (Type  I)  methods  using  heuristic  (d) 
above  "consistently  performed  very  well"  [2].  Similarly,  those  methods 
using  heuristic  (b)  were  found  to  perform  nearly  as  well  as  (d) . 

McBride  [15]  compares  the  directional  gradient  method  (d)  with 
an  approach  suggested,  but  not  tested  by  Greenberg  and  Rarick  [7].  The 
latter  method  uses  the  conflict  matrix  as  does  heuristic  (b) .  However,  it 
focuses  on  finding  a  maximum  embedded  identity  matrix  within  the  conflict 
matrix,  rather  than  using  the  conflict  matrix  to  determine  conflict 

3 

counts,  applying  a  specialization  of  the  preassigned  pivot  procedure  (P  ) 
normally  used  for  reinversion  [8],  McBride’s  results  indicate  that 
heuristic  (d)  is  significantly  faster.  However,  neither  method  consistently 
achieves  a  larger  GUB  set. 

McBride  also  comments  on  the  notion  of  a  "good"  GUB  set.  He 
finds  merit  in  selecting  a  set  of  GUB  rows  that  minimizes  the  non-zero 
build-up  in  the  representation  of  the  inverse  transformation  of  the  explicit 
basis,  during  actual  optimization.  Results  are  also  given  for  a  restricted 
GUB  set  selection  that  gives  priority  to  equality  constraints.  Since 
equality  constraints  are  always  binding  in  feasible  solutions,  the  subset 
of  the  basis  associated  with  binding  constraints,  or  kernel  [6]  is  expected 
to  have  fewer  explicit  non-zero  elements . 

Based  upon  the  results  in  these  papers,  and  on  independent  com¬ 
putational  experience  with  automatic  GUB  factorization  reported  by  Brown 
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and  Graves  [3] ,  the  present  research  initially  concentrated  on  those 
approaches  utilizing  the  two  most  successful  heuristics,  based  on 
conflict  (1.2,  II. 2)  and  directional  gradient  (II. 9,  II. 10). 

The  models  studied  in  this  report  are  of  a  larger  scale  and  include 
mixed  integer  problems  as  well  as  models  for  which  prior  GUB  row  sets 
have  been  manually  specified. 

Most  of  the  notation  and  labels  of  (2]  have  been  retained  here. 


mm#'- 
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4.  DETERMINATION  OF  THE  ELIGIBLE  SET 

The  implementation  of  GUB  in  simplex  algorithms  usually  admits 
only  +  1  as  non-zero  coefficients  in  the  GUB  rows.  In  linear  programming, 
a  column  scaling  ran  make  each  non-zero  element  in  a  GUB  row  +  1.  For 
variables  of  an  integer  or  mixed  integer  programming  problem,  the  colunns 
of  matrix  A  that  correspond  to  integer  variables  cannot  be  scaled  without 
inconvenience  for  other  optimization  functions  depending  upon  the  integrality 
condition.  Therefore,  non-zero  elements  in  columns  corresponding  to 
integer  variables  will  be  modified  only  by  row  scaling.  If  it  is  impossible 
to  obtain  the  necessary  +  1  non-zero  coefficients  by  row  scaling  and 
column  scaling  of  columns  corresponding  to  continuous-valued  variables, 
the  row  is  deemed  not  eligible  for  inclusion  in  a  GUB  set. 

It  is  an  objective  of  this  research  that  the  procedures  examined 
for  locating  a  GUB  set  in  a  linear  programming  problem  be  designed  to  be 
incorporated  as  an  automatic,  integral  part  of  a  contemporary  optimizing 
system  of  advanced  design. 

Each  method  is  implemented  as  a  feature  of  the  read  routine 
(written  to  accept  input  in  the  standard  MPS  format,  as  well  as  editing 
information  Indicating  integer  variables,  scaling,  and  known  prior  GUB 
structure) .  Each  method  automatically  examines  the  rows  of  the  input 
and  specifies  a  GUB  set.  The  appropriate  rows  and  columns  are  then 
scaled  as  necessary  to  obtain  the  proper  GUB  structure,  and  passed  on  to 
the  optimizing  portion  of  the  system.  (Note  that  the  editing  information 
places  conditions  that  must  be  satisfied  for  any  achievable  GUB  set.) 
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In  determining  the  set  of  eligible  rows,  the  following  factors 
have  to  be  considered. 

a.  Through  the  editing  process,  have  some  of  the  rows  been  dropped  from 
the  problem?  If  so,  these  "masked"  rows  are  not  eligible  for  inclusion 
in  the  GUB  structure  and  are  thus  dropped  from  the  set  of  eligible  rows. 

b.  Through  the  editing  process,  have  any  rows  been  predesignated  to  be  in 
the  GUB  structure?  (As  previously  mentioned,  large  segments  of  the 
constraints  can  often  be  selected  for  the  GUB  set  either  visually  or 
by  recognition  of  a  member  of  a  convenient  class  of  models.)  Any  rows 
that  conflict  with  these  rows  are  not  eligible  for  subsequent  inclusion. 

c.  All  rows  designated  "nonconstrained"  (which  include  the  objective 
function)  are  Ineligible  for  inclusion  in  the  GUB  structure.  All 
such  rows,  other  than  the  objective  function,  are  subsequently  handled 
independently  of  the  optimization. 

d.  If  there  are  any  integer-valued  variables,  an  additional  check  is 
performed.  A  row  in  the  GUB  set  must  eventually  be  capable  of  being 
scaled  to  +  1  non-zero  coefficients.  This  is  achieved,  if  necessary, 
through  a  combination  of  row  and  column  scaling.  However,  with  integer 
variables,  column  scaling  is  no  longer  advisable.  Therefore  any  row 
with  a  non-zero  element  in  integer  columns  that  is  not  a  +1  or  -1, 

or  capable  of  being  rendered  into  a  +  1  in  those  positions  through 
row  scaling  alone,  ust  be  marked  as  ineligible  for  inclusion  in  the 
GUB  structure.  Figure  4  gives  the  flow  chart  of  how  this  procedure 
is  implemented. 

Once  the  above  restrictions  have  been  considered,  the  resulting  set 
of  eligible  rows  is  then  available  for  search  in  order  to  construct  the 


desired  GUB  structure. 


5.  IMPLEMENTATION  OF  AUTOMATIC  GUB  HEURISTICS 


Conflict  Methods 

The  approaches  1.2  and  II. 2  employ  the  notion  of  a  conflict  measure 
for  each  row.  Consider  the  conflict  matrix,  M,  of  the  corresponding 
technological  matrix  A,  for  which  a  GUB  set  is  to  be  found.  An  individual 


element,  m^  is  1  if  row  i  and  row  k  of  the  original  matrix  have  at 


least  one  column  j  such  that  a^  #  0  and  a^  #0.  If  the  two  rows 
have  no  non-zero  coefficients  in  a  common  column  then  the  corresponding 


m^k  of  the  conflict  matrix  is  0.  Summing  across  a  row  of  the  conflict 


matrix  can  thus  give  the  measure  of  the  number  of  rows  plus  one  that  are 
in  conflict  with  a  given  row.  For  a  given  row,  this  sum  less  one  indicates 
exactly  how  many  other  rows  would  be  immediately  excluded  from  the  GUB 
set  by  inclusion  of  this  row,  called  the  row's  deletion  potzntJjxZ . 

Method  1.2  initially  places  all  the  eligible  rows  on  a  candidate 
list.  From  the  candiate  list,  individual  rows  are  selected  and  removed 
to  be  added  to  the  structure  set.  Other  rows  that  are  in  conflict  with 
the  selected  row  are  immediately  removed  from  the  candidate  list  and  dis-~ 
carded. 


The  heuristic  selects  those  rows  on  the  candidate  list  with  the 
mindjnum  deletion  potential  to  be  added  to  the  structure  set  first.  The 
selection  of  rows  for  the  structure  set  and  the  discarding  of  conflicting 
rows  continues  until  the  candidate  list  is  exhausted.  The  resulting 
structure  set  forms  a  GUB  set. 

A  modification'  to  the  above  heuristic  is  possible  which  bsie.ak& 
among  rows  sharing  the  minimum  deletion  potential  by  selecting  the  row 
having  the  most  non-zero  elements  for  inclusion  into  the  GUB  structure  set. 


The  program  used  to  test  this  heuristic  approach  is  adapted  from  an 
earlier  version  made  available  by  Graves.  A  step  by  step  description  of  the 
method  is  given  below. 


Step  1.  Identify  Eligible  Rows.  Set  8^  =  1  if  row  i  is  an  eligible  row, 
and  equal  to  0  otherwise. 

Step  2.  Determine  Deletion  Potential.  Scan  each  eligible  row  i  and  incre¬ 
ment  6.  by  the  number  of  other  eligible  rows  k  where  a,  and  a,  .  are 
i  lj 

both  non-zero  for  at  least  one  column  j.  (8^  is  the  deletion  potention, 
plus  one.) 

Step  3.  Stopping  Condition.  If  all  the  8^  *  0,  stop.  Otherwise,  go  to 
the  next  step. 


Step  4.  Row  Selection.  Select  row  i  having  the  minimum  positive  ("deletion 


potential")  8i  and  add  it  to  the  structure  set. 

Step  5.  Exclude  Rows  in  Conflict  with  Selected  Row.  Locate  the  (ei-i)  rows 
in  conflict  with  the  selected  row  i.  For  each  of  these  rows  k,  locate  the 


(gj^-l)  rows  that  they  are  in  conflict  with  and  decrement  8^  for  those 
rows  by  one. 

Step  6.  Marking  Selected  and  Excluded  Rows  Ineligible  for  Further 
Consideration.  Set  8^  and  the  equal  to  zero.  Go  to  step  3. 


Only  rows  with  8^  >  0  are  eligible.  In  step  1 ,  g ^  is  set  to  1  for  eligible 
rows.  In  the  next  step  the  s's  for  these  rows  are  modified  by  each  row's 


deletion  potential.  Assuming  there  are  still  some  eligible  rows,  the  one 
with  the  smallest  deletion  potential  is  selected  in  step  4 


22 


for  Inclusion  in  the  structure  set.  In  the  next  step  all  the  rows  con¬ 
flicting  with  the  one  selected  are  identified  for  discard  and  the  deletion 
potentials  of  the  remaining  rows  are  updated.  In  the  last  step,  both  the 
selected  row's  weight  and  those  of  the  discarded  rows  are  set  equal  to 
zero.  When  all  rows  have  either  been  selected  or  discarded  the  0  array 
will  be  all  0’s.  At  this  point  the  selected  rows  from  a  GUB  structure. 

Method  II. 2  (row  deletion)  initially  places  all  the  eligible  rows 
in  the  structure  set.  From  this  set  individual  rows  are  selected  and 
placed  on  the  candidate  list  in  order  of  maximum  deletion  potential. 

During  Phase  2,  Brearley,  Mitra,  and  Williams  drop  all  rows  from  further 
consideration  that  conflict  with  the  structure  set  and  attempt  to  reinclude 
remaining  candidate  rows  (that  do  not  conflict  with  the  structure  set)  in 
LOFI  order.  A  modification  of  phase  2  is  used  in  this  research  which 
simply  excludes  from  further  consideration  all  conflicting  rows,  reincludes 
any  remaining  candidate  rows,  and  repeats  phase  1,  until  no  further  non¬ 
conflicting  candidates  remain. 

Gradient  Methods 

The  second  method  (II. 9,  II .10)  employs  a  heuristic  method  put 
forth  by  Senju  and  Toyoda  [17]  for  approximate  solution  of  certain  linear 
programning  problems  with  0,  1  variables.  The  general  problem  they  address 
is  that  of  choosing  a  most  profitable  combination  (or  portfolio)  of  orders 
subject  to  resource  constraints  and  an  all-or-nothing  (0-1)  restriction 
on  the  orders  (i.e.,  an  order  is  not  allowed  to  be  only  partially  filled). 

This  same  format  can  be  used  to  express  the  search  for  a  maximum 
GUB  structure  in  (L) ,  The  rows  of  the  technological  matrix  A  are  treated 
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like  the  orders  in  the  Senju  and  Toyoda  model  in  that  they  are  to  be 
either  included  with  or  excluded  from  the  GUB  set.  The  objective 
is  to  obtain  a  maximum  number  of  rows  in  the  GUB  structure  while  satisfy¬ 
ing  the  stipulation  that  the  GUB  rows  be  disjoint.  This  last  restriction 
can  be  expressed  as  a  set  of  resource  restrictions  in  the  sense  of 
Senju  and  Toyoda. 

In  mathematical  terms ,  the  GUB  finding  problem  can  be  formulated 
as  follows: 

(S)  Max  Z  «  x.  +  x0  +•••+  x 
i.  t  m 

s.t.  I  kljXi  1  1  ,  j  -  1 . n 

where  x.^  e  {0,1} 

m  is  the  number  of  candidate  rows  in  (L) , 

n  is  the  number  of  variables  in  (L) , 

is  the  (i,j)  element  of  the  matrix  K,  which  in  turn 
is  the  0,  1  matrix  associated  with  the  matrix  A, 
x^  is  the  variable  which  determines  whether  row  i  is  in 
the  GUB  set  or  not, 

Z  is  the  objective  function. 

Senju  and  Toyoda  outline  a  heuristic  approach  for  obtaining  a 
near-optimal  solution  for  the  problem  they  examine.  Adapting  their 
approach  to  the  specialization  (S)  given  above,  a  type  II  strategy  results, 
with  all  the  rows  initially  being  Included  in  the  GUB  structure  set. 

Using  the  vector  space  viewpoint  outlined  earlier,  consider  each  row  of  (S) 
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as  a  vector  in  n-space.  A  resultant  vector  R  is  determined  by  the 
sum  of  all  the  Included  rows  and.  In  general,  extends  beyond  the  feasible 
space  denoted  by  the  unit  hypercube.  A  gradient  vector  Is  calculated 
from  this  Infeasible  point  In  the  direction  of  the  shortest  distance  to 
the  feasible  region.  In  Brearley,  Ultra  and  Williams  [2]  this  vector 
Is  labeled  a**.  An  Inner  product  of  this  gradient  with  each  of  the 
row  vectors  results  in  a  relative  weight  for  each  row.  These  weights, 
which  are  stored  in  a  vector  labeled  v**  can  be  viewed  as  Indicating 
the  relative  contribution  that  the  removal  of  the  corresponding  row 
would  have  towards  obtaining  a  feasible  structure  set. 

Rows  are  removed  from  the  structure  set  according  to  their 
relative  weight,  the  largest  weight  being  removed  first.  This  process 
is  continued  until  a  feasible  set  of  GUB  rows  has  been  obtained.  (The 
gradient  vector  is  not  recomputed  as  the  method  proceeds.) 

Next,  a  phase  2  procedure  is  implemented  which  examines  each  of 
the  initially  removed  rows  to  see  if  any  can  be  reincluded  into  the 
structure  set  without  violating  the  bounds  of  the  unit  hypercube.  Upon 
completion  of  phase  2,  the  selected  rows  constitute  a  GUB  set. 

A  variation  on  the  above  procedure  recalculates  the  shortest 
distance  to  the  feasible  region  after  the  removal  of  each  row.  With  the 
new  gradient,  a  new  set  of  relative  weights  for  the  remaining  rows  is 
then  calculated  and  used,  if  necessary,  to  determine  which  of  the  subsequent 
rows  will  be  removed.  This  method  is  named  II. 9. 

Another  modification  is  possible  if  two  rows  are  found  with  equal 
weights.  As  a  tie-breaking  rule,  the  row  found  to  have  the  least  number 
of  non-zero  coefficients  is  discarded  first. 


A  step  by  step  outline  of  the  heuristic  approach  II. 10  follows: 

Phase  1:  Deletion  of  Infeasible  Rows 


Step  0.  Initialize  Sets.  Add  all  eligible  rows  to  the  structure 
set.  The  candidate  set  is  empty. 

Step  1.  Determine  the  Vector  R.  For  each  column  j,  define  as 
the  number  of  rows  in  the  structure  set  having  non-zero  elements 
in  column  j . 

Step  2.  Determine  Relative  Weight  of  each  Row.  For  each  row  i, 
define  as  the  sum  of  (p^-l)  of  every  column  j,  for  which  a^  ¥  0. 

Step  3.  Feasibility  Condition.  If  for  every  column,  p^  <_  1,  then  go 

to  step  6;  else  find  a  column  j  such  that  p^  >  1. 

Step  4.  Determine  Row  for  Exclusion.  Examine  the  rows  in  the 
structure  having  non-zero  elements  in  column  j .  Select  the  row  i 
with  the  largest  v^. 

Step  5.  Remove  Selected  Row.  Remove  row  i  from  the  structure  set, 

decrementing  p^  by  one  for  every  column  j  with  a^  ¥  0.  Add  row  i 

to  the  candidate  set  and  return  to  step  3. 

Phase  2:  Improve  Feasible  GUB  Set  Found  by  Re-including  Excluded  Rows 

Step  6.  Eliminate  Rows  in  Candidate  Set  that  Conflict  with  the  Feasible  Set. 

For  every  row  i  of  the  candidate  set  that  has  at  least  one  a^  ¥  0 

in  a  column  with  *  1,  remove  that  row  from  the  candidate  set. 

Step  7.  Re-inclusion  Of  Rows.  If  any  rows  remain  in  the  candidate 
set,  then  find  row  i  having  the  smallest  v^.  Remove  row  i  from  the 
candidate  set  and  re-include  it  in  the  structure  set.  Increment  p^ 
by  one  for  every  column  j  where  a^  ¥  0. 

Step  8.  Stopping  Condition.  If  the  candidate  set  is  empty,  stop; 


else  go  to  step  6. 
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In  Seep  1,  Che  vector  p  is  calculated  as  the  column  counts  of  non* 
zero  coefficients.  Step  2  calculates  the  relative  weights  that  result  from 
the  inner  product  of  the  gradient  vector  with  each  of  the  row  vectors. 

These  are  stored  in  the  array  The  next  step  examines  p  to  see  if  the 

vector  is  within  the  feasible  region.  If  not*  a  row  with  the  largest  relative 
weight  is  removed  from  the  structure  set  and  the  p  vector  is  updated  to 
reflect  the  sum  of  the  row  vectors  remaining  in  the  structure  set. 

Once  a  feasible  structure  has  been  obtained,  the  candidate  set  (which 
consists  of  those  rows  initially  removed)  is  scanned  in  Step  6.  Any  of  those 
rows  found  to  still  be  in  conflict  with  the  rows  of  the  structure  set  are 
discarded.  Among  those  rows  which  remain,  that  with  the  smallest  relative 
weight  is  re-included  in  the  structure.  This  cycle  of  discarding  and  re- 
inclusion  is  continued  until  the  candidate  set  has  been  emptied.  The  result¬ 
ing  rows  of  the  structure  set  constitute  a  feasible  GUB  set. 

To  modify  the  algorithm  in  order  to  compute  a  new  gradient  vector 
after  the  removal  of  each  row  in  phase  1,  step  5  is  changed  as  follows  (II. 9): 

Step  5.*  Remove  Selected  Row.  Remove  row  i  from  the  structure. 


decrementing  p^  by  one  for  every  column  j  such  that  a^  j*  0.  Locate 
each  row  k  that  is  in  conflict  with  row  i.  Decrement  by  the 

number  of  conflicts  between  the  two  rows.  Add  row  i  to  the  candidate 


set  and  return  to  step  3. 


Now  when  a  row  is  removed  from  the  structure  set,  the  contain 

the  new  relative  weights  equal  to  the  inner  product  between  the  vector  for 


row  1  and  the  new  gradient. 
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These  two  basic  methods  have  been  implemented  as  integral  modules 
of  a  large  scale  optimization  system.  Therefore,  explicit  conflict  matrices 
are  not  built.  (To  have  done  so  would  have  consumed  too  much  computer 
time  and  space.)  Instead,  all  the  information  is  stored  in  the  vectors  8, 
p,  and  v.  Logical  flags  associated  with  each  row  indicate  whether  it  is 
eligible,  and  whether  it  is  in  the  candidate  set  or  in  the  structure  set. 

As  mentioned  previously,  the  problem  data  is  read  in  MPS  format  and 
expressed  internally  in  terms  of  only  the  non-zero  elements.  This  input  is 
stored  in  a  doubly  linked  list  having  both  a  row  and  a  column  thread.  Thus, 


along  with  any  non-zero  coefficient  a^  ,  the  location  of  adjacent  non-zero 
eleo^nts  in  both  the  row  1  and  column  j  are  also  immediately  available.  This 


crucial  feature  permits  efficient  row  access  for  various  operations  (e.g.. 


to  locate  all  rows  that  conflict  with  a  given  row  at  a  particular  column) . 
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6.  COMPUTATIONAL  RESULTS 


The  heuristic  methods  have  been  tested  on  15  real-life  problems  that 
vary  In  size  from  92  constraints  to  4,648  constraints.  A  description  of  each 
of  the  problems  Is  given  in  Figure  5.  As  can  be  seen,  four  of  the  problems 
are  mixed  Integer  and  two  are  pure  Integer. 


Problem 

Number 
of  rows' 

Number 
of  columns 

Integer 

Columns 

Non-Zeros 

VANN 

92 

1,324 

1,324 

2,648 

NETTING 

103 

247 

103 

494 

AIRLP 

171 

3,040 

0 

6,023 

COAL 

171 

3,753 

0 

7,506 

TRUCK 

239 

4,752 

4,752 

30,074 

CUPS 

415 

619 

145 

1,341 

PERT 

606 

9,024 

0 

40,484 

PIES 

663 

2,923 

0 

13,288 

PAD 

695 

2,934 

0 

13,459 

ELEC 

785 

2,800 

0 

8,462 

GAS 

799 

5,536* 

0 

27,474 

FOAM 

1,017 

4,020 

42 

17,187 

LANG 

1,236 

1,425 

0 

22,028 

JCAP 

2,487 

3,849 

560 

9,510 

ODSAS 

4,648 

'4,683 

0 

30,520 

Figure  5 


The  results  of  these  experiments  are  given  in  Appendix  A.  The  first 
two  columns  give  the  rows  and  non-zero  column  elements,  respectively,  of  the 
GUB  structures  found.  The  time  given  in  coluntt  three  is  the  time  required  to 


locate  the  GUB  set  once  the  set  of  eligible  rows  has  been  determined.  The 
final  columns  give  additional  information  relating  to  the  two  versions  of 
the  gradient  methods  examined  and  represents  total  time  in  phase  1  and  the 
number  of  rows  re-included  in  the  GUB  structure  during  phase  2.  All 
execution  times  reported  are  expressed  in  actual  CPU  seconds,  accurate  to  the 
precision  displayed,  for  IBM  360/67  and  FORTRAN  H  (Extended) . 

As  with  the  earlier  work  cited,  the  Senju  and  Toyoda  method  was  found 
to  be  consistently  the  fastest.  In  general  this  holds  true  for  both  method 
II. 9  and  II. 10.  II .9,  which  updates  the  gradient  after  each  row  is  removed, 
takes  longer  in  phase  1  than  its  counterpart.  However,  it  so  selectively 
deletes  the  rows,  that  few  if  any  rows  are  ever  added  back  into  the  structure 
during  phase  2.  This  suggests  the  possibility  of  implementing  II. 9  as 
only  a  one  phase  method. 

All  methods  are  robust  in  that  they  consistently  find  large  GUB  sets. 
The  conflict  approaches  generally  find  a  larger  number  of  variables  with 
non-zero  coefficients  in  the  GUB  rows.  However,  this  approach  definitely 
becomes  inefficient  when  larger  problems  are  analyzed,  regardless  of  the 
relative  size  of  the  GUB  structure  in  the  problem. 

There  is  some  discrepancy  between  these  results  and  those  published 
earlier  [2],  especially  with  regard  to  the  times  of  the  other  methods  com¬ 
pared  to  II. 10.  The  wide  discrepancy  between  II. 9  and  II. 10  has  not  been 
observed  in  the  current  experiments.  It  is  hypothesized  that  this  is  due 
partially  to  differences  in  implementation  of  the  various  approaches  and 
partially  to  problem  size  and  structure  variations  between  these  studies. 


7.  PROBLEM  COMPLEXITY 


The  oomptextty  of  a  problem  Is  said  to  be  polynomial  if  an  algorithm 
exists  for  which  the  fundamental  operations  are  limited  by  a  polynomial  func¬ 
tion  of  intrinsic  problem  dimensions.  Such  an  algorithm  would  be  called  a 
potynomiat  time,  or  good  algorithm.  The  class  of  all  problems  for  which  such 
algorithms  exist  is  denoted  (P) .  If  an  algorithm  is  not  polynomial  time, 
then  it  is  defined  to  be  an  e.k.pone.ntiat  time,  algorithm.  The  disadvantage 
of  an  exponential  algorithm  is  the  explosive  growth  of  the  maximum  solution 
time  as  the  dimensions  of  the  problem  Increase  [13]. 

A  problem  x  is  said  to  be  tizdu&Lbte.  to  a  problem  y  if  each  good 
algorithm  for  solving  y  can  be  used  to  produce  in  polynomial  time  a  good 
algorithm  for  solving  x  [11],  Note  that  this  does  not  necessarily  require 
that  a  good  algorithm  for  x  and  y  actually  exist.  This  requires  only 
that  if  one  exists  for  y,  then  one  also  exists  for  x. 

An  tnt/uicXabte.  problem  is  one  for  which  it  is  known  that  no  polynomial 
time  algorithm  exists.  In  between  this  class  of  problem,  and  the  class  P, 
is  a  vast  number  of  problems  whose  status  is  uncertain.  Among  these  is  a 
class  of  nondLteAmintitic  potynomiat- time,  problems  (NP)  for  which  a  poly¬ 
nomial  time  algorithm  can  be  shown  to  exist  that  can  veJiLfay  a  guessed  solu¬ 
tion,  but  for  which  the  existence  of  a  (deterministic)  polynomial  time 
algorithm  to  actually  solve  a  problem  has  not  yet  been  demonstrated. 

If  every  problem  of  the  class  NP  is  reducible  to  the  problem  y, 
then  y  is  said  to  be  KP-haAd.  In  addition,  if  y  itself  belongs  to  NP, 
then  y  is  NP-c omptete.  [5,11]. 


The  following  problem  is  known  as  the  ■Cnd&pe.ndznt  &e£  dzcAA-ion  psiobZe/n 
(ISO).  It  belongs  to  the  set  of  NF-complete  problems. 

(ISD)  Given  a  graph  G  ■  (V,E)  and  an  integer  t,  decide  whether  G  con¬ 
tains  an  independent  set  of  size  t  or  more. 

The  GUB  decision  problem  (GUBD)  can  be  defined  as  follows: 

(GUBD)  Given  an  m  *  n  0,1  matrix  K  and  an  integer  p,  decide  whether  K 
contains  a  set  of  p  or  more  rows  i^»  ^  »  i  such  that 

(*)  f  k.  ,  <_  1  for  every  column;  q  >_  p. 

e-1  eJ 

Given  an  instance  of  the  ISO  problem,  the  incidence  matrix  N  can  be  con¬ 
structed.  This  matrix  along  with  the  integer  t  is  an  instance  of  the  GUBD 
problem.  The  following  theorem  proves  the  correctness  of  this  reduction: 

Theorem:  The  incidence  matrix  N  has  t  rows  satisfying  (*)  if  and  only 
if  there  are  t  vertices  in  G  that  are  independent. 


Proof. 

a)  Assume  there  exists  t  rows  of  N  that  satisfy  (*) .  They  correspond 

to  vertices  v.  ,v  ,  ...  ,  v .  in  G.  If  any  two  of  these  vertices 

11  12  t 

are  adjacent,  then 


where  j  is  the  column  in  N  that  corresponds  to  the  edge  connecting 
the  two  vertices.  This  is  a  violation  of  the  assumption,  hence  die 


t  vertices  in  G  are  not  connected  to  one  another. 
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b.  Assume  there  exists  t  vertices  v.  ,  v.  ,  ...  ,  v.  In  G  that  are 

11  12 

Independent.  Since  no  two  are  adjacent,  the  corresponding  rows  in  N 
satisfy  (*)  [20].  Q.E.D. 

Since  the  ISD  problem,  a  problem  known  to  be  NP-complete,  is  reducible 
to  the  GUBD  problem,  it  follows  that  the  GUBD  problem  itself  is  NP-complete. 
(It  is  clear  that  the  reduction  is  polynomial  time  and  it  is  also  clear  that 
GUBD  is  in  NP.)  The  related  problems  of  finding  a  maximum  independent  set 
and  a  maximum  GUB  set  are  not  in  NP,  however,  they  are  NP-hard.  It  is  there¬ 
fore  unlikely  that  a  polynomial- time  algorithm  will  be  found  for  these 
problems.  Only  exponential-time  algorithms  are  presently  available. 

The  above  analysis  of  GUB  algorithms  has  only  addressed  the  vooHAt  COAZ 
bound.  No  conclusions  are  made  about  the  average  performance  of  an  algorithm. 
In  other  words,  the  possibility  of  the  existence  of  an  algorithm  with  good 
average  performance,  but  having  an  exponential  worst  case  bound,  has  hot 
been  ruled  out. 
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8.  AN  UPPER  BOUND  FOR  THE  SIZE  OF  MAXIMUM  GUB  SET 


The  intrinsic  difficulty  of  identifying  a  maximum  GUB  set  has  been 
shown  to  be  exponential,  making  this  task  essentially  impossible  for  problems 
of  the  scale  at  hand.  However,  the  efficient  heuristic  procedures  have  been 
shown  to  provide  very  large  GUB  sets,  whose  size  appears  to  be  relatively 
stable  for  each  problem  regardless  of  the  particular  method  applied.  This 
suggests  that  these  large  GUB  sets  may  be,  in  fact,  very  nearly  maximum, 
although  there  is  no  practical  way  to  verify  this  directly. 

Although  the  problem  of  determining  the  size  of  the  maximum  GUB  set 
is  also  NP-hard,  it  is  possible  to  develop  an  easily  computable  uppeA.  bound  on 
the  maximum  GUB  set  size .  This  bound  can  then  be  used  to  objectively 
evaluate  the  quality  of  the  GUB  sets  produced  by  heuristic  algorithms. 

It  is  clear  that  the  number  of  rows  of  a  GUB  set  can  be  no  greater 
than  the  number  of  rows  in  the  problem.  Also  any  one  row  by  itself  can  form 
a  GUB  set.  But  these  bounds  are  of  little  practical  use  where  considering 
the  problem  of  identifying  a  maximum  GUB  set.  Utilizing  information  that 
is  already  available  in  the  heuristic  procedure,  it  is  possible  to  construct 
in  polynomial  time  an  upper  bound  on  the  size  of  the  maximum  GUB  set.  (It 
is  also  possible  to  construct  a  lower  bound  on  the  size  of  the  maximum  GUB 
set,  but  that  topic  is  not  pursued  in  this  report.) 

For  the  purpose  of  developing  a  better  bound,  the  incidence  matrix 
representation  (N)  of  the  problem  is  used.  Let  si  be  the  number  of  l's 
in  row  i.  Note  that  s^  is  the  number  of  edges  incident  to  vertex  i  in  G. 
Also  note  that  s^  ■  fs,,-l.  The  number  of  columns  in  N  represents  the 
nunfeer  of  distinct  conflicts  that  exist  between  the  rows  of  the  original 
problem.  This  number  is  denoted  as  c,  and  can  be  found  by  the  following 


formula. 
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C  ■ 


m 

2 

If  c  Is  greater  than  0,  all  the  rows  of  N  cannot  simultaneously  belong 
to  a  GUB  set,  which  Implies  the  cardinality  of  the  GUB  set  Is  less  than  m. 
As  c  becomes  larger,  the  following  argument  shows  that  the  upper  bound  of 
the  maximum  GUB  set  decreases. 

If  c  is  positive,  but  strictly  less  than  m,  it  Is  possible  for  all 
the  conflicts  to  Involve  one  row.  Removal  of  that  row  would  then  leave  m-1 
rows  that  form  a  GUB  set.  Thus  for  c  in  the  range  from  1  to  m-1,  an  upper 
bound  on  the  size  of  the  maximum  GUB  set  is  m-1.  Since  one  row  can  conflict 
with  at  most  m-1  other  rows,  once  c  m,  at  least  two  rows  have  to  be  re¬ 
moved  to  form  a  GUB  set.  For  m  £  c  [(m-1)  +  (m-2)]  it  is  possible  to 
construct  a  incidence  matrix  such  that  all  the  conflicts  are  between  a  pair 
of  rows  and  the  remaining  set  of  rows.  Removal  of  the  pair  would  result  in 
a  GUB  set  of  m-2  rows.  This  constructive  argument  continues  until 
c  ■  [(m) (m-1) ]/2,  which  occurs  when  each  row  conflicts  with  every  other  row. 
At  that  point,  the  max  maximum  GUB  *  min  maximum  GUB  ■  one  row. 

A  graph  of  an  upper  bound  on  the  maximum  GUB  for  a  five  row  problem 
such  as  the  example  problem  is  given  below: 


Figure  6 
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For  the  example  problem,  m  ■  5  and  c  ■  5.  From  the  above  graph  the  upper 
bound  on  the  maximum  GUB  for  that  problem  is  3.  Since  a  GUB  set  containing 
three  rows  has  already  been  identified,  that  set  is  a  maximum  set. 

In  general,  for  any  problem  with  an  m  x  c  incidence  matrix,  the 
largest  maximum  GUB  set  that  can  be  obtained  is: 

u^  *  .5  +  /.25  +  (m)  (m-1)  -  2c 

where  L  indicates  truncation  to  an  integer. 

The  above  bound  is  pAobtZm-<ndzpZnd£.nt  and  a  ikaup  bound  in  that 
matrices  with  a  GUB  set  the  size  of  the  bounding  value  can  be  constructed. 

With  additional  information  about  a  specific  problem  a  better  bound 
can  be  constructed.  Since  s^  is  the  number  of  other  rows  that  conflict 
with  row  1,  removing  row  i  from  the  set  of  rows  reduces  the  number  of  con¬ 
flicts,  c,  by  s^.  Let  IMAX  denote  max  s^.  Since  IMAX  is  the  largest 
row  conflict  count,  c  can  be  reduced  by  no  more  than  IMAX  with  the  removal 
of  each  row.  The  minimum  number  of  rows  that  would  have  to  be  removed  to 
reduce  the  number  of  row  conflicts  to  0,  is  T c/IMAX.  Therefore,  given  m, 
c  and  IMAX,  the  bound  can  be  improved  to 


u 


2 


m  - 


» 


L 


.5  +  /.25  +  y (2m-y-l)  -  2c  , 


c  _<  (m-y)y 


c  >  (m-y)  (y)  ; 


where  y  *  IMAX  and  |  indicates  the  next  higher  integer. 

In  order  to  determine  IMAX,  the  entire  6  vector  must  be  examined. 

A  third,  even  better  bound  can  be  obtained  with  additional  information 
on  the  of  the  conflict  counts  from  1  to  IMAX.  The  procedure  is 

the  same  as  above,  in  that  when  a  row  is  removed  with  IMAX  conflict  count. 
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c  decreases  by  IMAX.  However,  instead  of  continuing  to  decrease  c  by 
IMAX,  it  is  decreased  by  the  next  largest  This  procedure  continues 

until  once  again,  c  becomes  zero.  This  bound  is  named 

Each  tighter  bound  requires  more  information  about  the  particular 
problem.  However,  all  the  information  is  readily  available  since  it  is 
generated  by  the  heuristics  using  the  conflict  measure. 

The  bounds  developed  can  be  used  to  objectively  evaluate  the  size 
of  a  GUB  set  found  by  heuristic  methods.  In  two  problems  examined,  VANN 
and  AIRLP,  the  number  of  rows  in  the  GUB  set  equal  an  upper  bound  on  the 
maximum  GUB  set  for  the  problem.  Therefore,  for  those  problems,  the  heuristic 
methods  are  verified  to  have  located  maximum  GUB  sets. 

Manual  specification  of  a  GUB  set  from  visual  inspection  can  utilize 
these  bounds  as  an  excellent  measure  of  the  maximum  additional  rows  to  be 
found.  This  information  is  also  an  aid  in  deciding  whether  to  subject  the 
problem  to  additional  automatic  searching  for  GUBs. 
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9.  EXTENSIONS 


The  upper  bounds  developed  in  this  report  vary  from  a  problem-inde¬ 
pendent  bound  to  tighter  problem-dependent  bounds.  It  is  speculated  that 
additional  information  can  be  easily  extracted  from  the  actual  conflict 
structure  of  the  problems  that  can  be  used  to  tighten  the  existing  bounds 
even  further.  This  is  strongly  suggested  by  manual  analysis  of  problems  with 
particularly  loose  bounds  for  which  the  conflict  structure  seems  to  have  higher 
order  pathology.  In  addition,  lower  bounds  have  been  developed  by  similar  methods. 

Another  area  that  warrants  further  study  is  the  special  structure 
of  the  incidence  matrix  representation  of  the  original  problem.  It  is  noted 
that  for  an  incidence  matrix,  N,  the  relative  weights  generated  for  each  row 
are  (except  for  a  constant)  identical  for  the  methods  studied.  This  implies 
that  for  a  matrix  N,  and  the  same  strategy  (i.e.,  II,  deletion),  the  two 
heuristics  would  identify  the  same  GUB  set. 

As  things  now  stand,  GUB  finding  demands  far  less  cost  than  the 
benefits  derived  during  model  optimization.  Better  GUB  finding  methods  may 
result  from  simple  extensions  arising  from  relaxations  of  (S),  use  of  con¬ 
flict  information  of  higher  order,  limited  application  of  backtracking 
enumeration,  or  exploitation  of  conditioned  bounds  on  the  remaining 
candidate  rows  to  allocate  heuristic  effort. 

Finally  research  is  continuing  on  automatic  location  of  network  row 
structure  (e.g.,  Sarquis  [16]  and  Wright  [19]).  As  one  illustration  of  an 
itmediate  generalization  of  the  GUB  results,  a  GUB  set  for  a  problem  can 
be  identified  and  then  another  GUB  set  of  an  eligible  subset  of  remaining 
rows  can  be  found.  Thus,  a  b-L-panXiXx.  neJwotik  MM  ^actoxizaZLon  can  be 
achieved  (e.g.,  transportation  or  assignment  rows). 
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10.  CONCLUSIONS 


The  computational  benefits  of  a  large  GUB  set  for  an  LF  problem  are 
widely  recognized.  This  report  shows  that  the  identification  of  a  maximum 
GUB  set  Is  a  difficult  problem,  essentially  as  hard  as  many  other  widely 
known  difficult  problems. 

An  alternate  approach  is  the  use  of  an  heuristic.  This  report  has 
examined  two  promising  methods  (with  two  versions  of  each)  with  application 
to  a  series  of  real  life,  large  scale  models.  All  versions  are  robust  in 
their  ability  to  find  large  GUB  row  sets.  However,  the  two  versions 
(II. 9  and  II. 10)  that  use  the  Senju  and  Toyoda  method  are  consistently 
the  fastest.  These  two  methods  are  essentially  equal  in  their  efficiency 
and  effectiveness.  Since  version  II. 9  (which  recalculates  the  gradient 
after  the  removal  of  each  row)  so  selectively  removes  the  rows  during 
the  first  phase  that  few  if  any  rows  are  re-included  in  the  GUB  set  during 
the  second  phase,  it  suggests  the  possibility  of  implementing  this  version 
as  only  a  one  phase  (row  deletion)  method. 

The  representation  of  an  infinite  number  of  m-row  matrices  by  a 
finite  nunber  of  incidence  matrices  offers  a  powerful  and  concise  way  of 
examining  the  GUB  problem.  Under  this  representation,  both  basic  heuristic 
methods  investigated  assign  (within  a  constant)  the  same  relative  selection 
weights  to  each  row. 

Finally,  the  ability  of  defining  upper  bounds  on  the  maximum  size 
of  the  GUB  set  gives  a  new  powerful  tool  in  this  area.  It  enables  one  to 
evaluate  the  quality  of  GUB  sets  found  even  in  very  large  problems,  for  which 
the  algorithmic  identification  of  a  maximum  GUB  set  is  probably  impossible 


in  general.  In  some  cases,  verification  of  a  heuristically  achieved  maximum 
GUB  set  is  now  possible.  Further,  the  bounds  developed  may  be  further  en¬ 
hanced  in  future  research,  and  may  be  applicable  to  related  problems  of 
equivalent  complexity. 
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APPENDIX  A 


This  appendix  contains  the  computational  results  for  the  fifteen 
linear,  mixed  integer  and  integer  models  examined.  All  execution  times 
reported  are  expressed  in  actual  CPU  seconds,  accurate  to  the  precision 
displayed  for  IBM  360/67  and  FORTRAN  H  (Extended) . 

For  clarity,  the  following  terms  are  defined: 

Eligible  rows:  The  number  of  rows  of  the  model  initially  eligible 
for  inclusion  in  a  set  of  GUB  rows. 

Conflict  count:  The  nimfcer  of  columns  of  the  incidence  matrix  for 
the  problem. 

Conflict  density:  The  ratio  of  the  conflict  count  to  the  maximum 

conflict  count  for  that  problem  size  [i.e.,  m(m-l)/2]. 
Time  to  find  Elig:  The  time  in  CPU  seconds  to  determine  the  set 

of  eligible  rows. 
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Problem 

VANN 

Description 

-Fleet  Dispatch  Model 

Rows 

92 

Eligible  rows 

69 

1MAX 

0  . 

Columns 

1324 

Conflict  count 

0 

U1 

69 

Integer 

1324 

Conflict  density 

0 

U2 

69 

Non-zero 

2648 

Time  to  find  Elig 

.141  sec 

U3 

69 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in 

Phase  2 

L2 

69 

1324 

.237 

H.2 

69 

1324 

.125 

H.9 

69 

1324 

.202 

.198 

0 

ilio 

69 

1324 

.202 

.198 

0 

Problem 

NETTING 

Description 

Currency  Exchange  Model 

Rows 

103 

Eligible  rows 

71 

IMAX 

5 

Columns 

247 

Conflict  count 

46 

U1 

70 

Integer 

103 

Conflict  density 

1.85 % 

U2 

59 

Non-zero 

494 

Time  to  find  Elig 

.022  sec 

U3 

46 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in 

Phase  2 

1.2 

36 

84 

.169 

H.2 

36 

84 

.164 

H.9 

36 

77 

.047 

.042 

0 

n.io 

36 

72 

.042 

.037 

0 

Problem 

AIRLP 

Description 

Fleet  Dispatch 

Model 

Rows 

171 

Eligible  rows 

170 

IMAX 

150 

Columns 

3040 

Conflict  count 

2983 

U1 

151 

Integer 

0 

Conflict  density 

20.77% 

U2 

150 

Non-zero 

6023 

Time  to  find  Elig 

.076  sec 

U3 

150 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in 

Phase  2 

L2 

160 

3000 

1.16 

IL2 

150 

3000 

.761 

0.9 

150 

3000 

.645 

.639 

0 

ILIO 

150 

3000 

.444 

.439 

0 
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Problem 

COAL 

Description 

Energy  Development  Model 

111 

Rows 

171 

Eligible  rows 

170 

IMAX 

Columns 

3753 

Conflict  count 

3753 

U1 

146 

Integer 

0 

Conflict  density 

26.13% 

U2 

136 

Non-zero 

7506 

Time  to  find  Elig 

.106  sec 

U3 

121 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase 

2 

1.2 

111 

3753 

1.38 

II.2 

111 

3753 

1.24 

0.9 

111 

3753 

.920 

• 

912 

0 

11.10 

100 

2568 

.641 

.631 

0 

Problem 

TRUCK 

Description 

Fleet  Dispatch  Model 

Rows 

239 

Eligible  rows 

221 

IMAX 

171 

Columns 

4752 

Conflict  count 

10438 

U1 

165 

Integer 

4752 

Conflict  density 

42.94% 

U2 

159 

Non-zero 

30074 

Time  to  And  Elig 

.116  sec 

U3 

144 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  i. 

dded 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase 

2 

1.2 

32 

1069 

6.88 

IL2 

30 

1099 

7.095 

n.9 

30 

857 

5.00 

4.95 

2 

n.io 

32 

986 

1.70 

1.58 

8 

Problem 

CUPS 

Description 

Production  Scheduling  Model 

Rows 

415 

Eligible  rows 

390 

IMAX 

48 

Columns 

619 

Conflict  count 

744 

U1 

388 

Integer 

145 

Conflict  density 

.98% 

U2 

374 

Non-zero 

1341 

Time  to  find  Elig 

.042  sec 

U3 

294 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUB  set 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase 

2 

L2 

213 

494 

2.96 

IL2 

214 

442 

3.15 

n.9 

214 

466 

.212 

.194 

0 

n.io 

200 

394 

.384 

.132 

24 

n.io 
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Problem 

FERT 

description 

Production  &  Distribution  Model 

Rows 

606 

Eligible  rows 

605 

IMAX 

580 

Columns 

9024 

Conflict  count 

16455 

U1 

677 

Integer 

0 

Conflict  density 

9.01% 

U2 

576 

Non-zero 

40484 

Time  to  find  Elig 

.257  sec 

U3 

567 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUB  set 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase  2 

12 

559 

9024 

15.8 

n 2 

559 

9024 

10.5 

n.9 

559 

9024 

6.73 

6.71 

0 

IL10 

559 

9024 

2.52 

2.50 

0 

Problem 

PIES 

Description 

Energy  Production  &  Consumption  Model 

Rows 

663 

Eligible  rows 

662 

IMAX 

21 

Columns 

2923 

Conflict  count 

4116 

U1 

655 

Integer 

0 

Conflict  density 

1.88% 

U2 

466 

Non-zero 

13288 

Time  to  find  Elig 

.866  sec 

U3 

422 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in  . 

Number  added 

GUB  set 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase  2 

L2 

180 

1848 

10.8 

n.2 

169 

1693 

13.5 

• 

n.9 

172 

1811 

2.82 

2.77 

1 

ilio 

177 

1761 

1.31 

788 

28 

Problem 

PAD 

Description 

Energy  Production  &  Consumption  Model 

Rows 

695 

Eligible  rows 

694 

IMAX 

23 

Columns 

2934 

Conflict  count 

4416 

U1 

687 

Integer 

0 

Conflict  density 

1.84% 

U2 

502 

Non-zero 

13459 

Time  to  find  Elig 

.104  sec 

U3 

449 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUB  set 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase  2 

L2 

200 

1864 

13.1 

IL2 

189 

1771 

16.6 

n.9 

188 

1708 

3.34 

3.26 

2 

n.io 

189 

1275 

1.35 

.928 

21 
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4» 


4 


Problem 

ELEC 

Description 

Energy  Production  &  Consumption  Model 

Rows 

785 

Eligible  rows 

784 

IMAX 

22  . 

Columns 

2800 

Conflict  count 

6167 

U1 

776 

Integer 

0 

Conflict  density 

2.01% 

U2 

503 

Non-zero 

8462 

Time  to  find  Elig 

.089  sec 

U3 

492 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in 

Phase  2 

1.2 

309 

2461 

11.4 

UJ2 

210 

2791 

16.1 

D.9 

309 

2641 

1.15 

1.12 

0 

IL10 

309 

2605 

.842 

.579 

14 

Problem 

GAS 

Description 

Production  Scheduling  Model 

Rows 

799 

Eligible  rows 

789 

IMAX 

608 

Columns 

5536 

Conflict  count 

22220 

U1 

760 

Integer 

0 

Conflict  density 

7.15% 

U2 

752 

Non-zero 

27474 

Time  to  find  Elig 

.151  sec 

U3 

652 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in 

Phase  2 

L2 

583 

5102 

16.2 

H.2 

639 

5536 

10.4 

n.9 

608 

5309 

3.79 

3.77 

0 

n.10 

639 

5533 

1.47 

1.44 

1 

Problem 

FOAM 

Description 

Production  Scheduling  Model 

Rows 

1017 

Eligible  rows 

1006 

IMAX 

261 

Columns 

4020 

Conflict  count 

8186 

U1 

997 

Integer 

42 

Conflict  density 

1.62% 

U2 

974 

Non-zero 

17187 

Time  to  find  Elig 

225 

U3 

934 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in 

Phase  2 

L2 

932 

4020 

23.4 

JL2 

932 

4020 

9.47 

n.9 

917 

3981 

1.73 

1.71 

0 

n.10 

917 

•3981 

.902 

879 

0 
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Problem 

LANG 

Description 

Equipment  &  Manpower  Scheduling  Model 

Rows 

1236 

Eligible  rows 

1235 

IMAX 

184 

Columns 

1425 

Conflict  count 

46424 

U1 

1196 

Integer 

0 

Conflict  density 

6.09% 

U2 

982 

Non-zero 

22028 

Time  to  find  Elig 

.072  sec 

U3 

973 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase  2 

L2 

382 

1207 

46.2 

U2 

338 

908 

54.2 

n.9 

342 

923 

14.9 

14.8 

2 

IL10 

342 

922 

12.4 

1.13 

234 

Problem 

JCAP 

Description 

Production  Scheduling  Model 

Rows 

2487 

Eligible  rows 

2446 

IMAX 

488 

Columns 

3849 

Conflict  count 

16578 

U1 

2439 

Integer 

560 

Conflict  density 

.55% 

U2 

2412 

Non-zero 

9510 

Time  to  find  Elig 

.265  sec 

U3 

1812 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase  2 

L2 

529 

2072 

104 

n.2 

512 

2186 

153 

• 

n.9 

529 

2087 

2.23 

1.87 

5 

n.io 

523 

1393 

3.98 

1.10 

59 

Problem 

ODAS 

Description 

Manpower  Planning  Model 

Rows 

4648 

Eligible  rows 

4647 

IMAX 

4194 

Columns 

4683 

Conflict  count 

5220 

U1 

4645 

Integer 

0 

Conflict  density 

.05% 

U2 

4645 

Non-zero 

30520 

Time  to  find  Elig 

.263  sec 

U3 

4024 

Method 

Rows  in 

Columns  in  Time  to  find 

Time  in 

Number  added 

GUBset 

GUB  set  GUB  set  (sec.) 

Phase  1 

in  Phase  2 

L2 

761 

3116 

369 

JL2 

721 

3846 

651 

0.9 

749 

4436 

7.12 

6.88 

0 

n.10 

751 

3020 

3.01 

2.57 

2 
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